Skip to content

Conversation

@AlishThapa
Copy link

@AlishThapa AlishThapa commented Sep 18, 2025

📦 Pull Request Template

📋 Summary

This pull request introduces a new defense mechanism, GraphPruningDefense, to the PyGIP framework.

This PR includes:

  1. GraphPruningDefense module, which is a compliant defense class, follows the IMPLEMENTATION.md
  2. Evaluation Script runs the defense on Cora, CiteSeer and PubMed datasets. If the model accuracy drops below 2%, then the optimal pruning ratio is determined and calculated.

✅ Checklist

  • My code follows the project's coding style
  • I have tested the changes and verified that they work
  • I have added necessary documentation (if applicable)
  • I have linked related issues above (if any)
  • The PR is made from a feature branch, not main

🧠 Additional Context (Optional)

pruning_effective_range_summary

@Edlison Edlison changed the base branch from main to dev September 30, 2025 01:24
@Edlison
Copy link
Contributor

Edlison commented Sep 30, 2025

Could you please complete the implementation by adding the metrics from the original paper, namely TCA, ECA, TBA, EBA, and fidelity?

@AlishThapa
Copy link
Author

AlishThapa commented Oct 3, 2025

I've completed the work on this PR. This includes the implementation of the GraphPruningDefense and a new, comprehensive evaluation for the existing BackdoorWM defense. Here’s a summary of the changes.

Graph Pruning Defense Enhancements
My initial task was to add the metrics TCA, ECA, TBA, EBA, and Fidelity to the GraphPruningDefense.

During development, I found that the TCA, TBA, and EBA are specific to backdoor-based defenses and they are not applicable to the graph pruning mechanism, which only removes edges.

Therefore, I have refactored the GraphPruningDefense and its evaluation script to focus on the metrics that are meaningful for this context:

  • Test Accuracy (ECA): The primary performance metric for the pruned model.
  • Fidelity: A new and valuable metric that measures how the model's predictive behavior changes as edges are removed.

The updated evaluation script (evaluate_graph_pruning.py) now generates an improved summary plot that visualizes both accuracy and fidelity, providing much richer insights into the effects of pruning.

New: Backdoor Watermarking Evaluation
I found that those metrices are applicable to the backdoor watermarking defense and the defense mechanism was already mentioned in the defense folder.
And to fulfill the original request of implementing all the specified metrics, I've added a new evaluation script for the pre-existing BackdoorWM.py defense.

A comprehensive summary plot that visualizes the final results across multiple datasets is also added.

In summary, this PR now delivers two distinct experiments: a refined analysis of Graph Pruning and a new, robust evaluation pipeline for Backdoor Watermarking.

Graph pruning data visualization.
pruning_ECA_fidelity_summary

Backdoor watermarking data visualization.
full_defense_analysis_summary

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants